Floating image display

ABSTRACT

Examples are disclosed that relate to optical systems. One example provides a display device comprising an image source including a plurality of encoded regions from which encoded image light is output, and a Fourier transform array. The Fourier transform array may be positioned to receive the encoded image light and output decoded image light that forms a floating image viewable from a plurality of different vantage points, wherein from a first vantage point decoded image light forming a portion of the floating image originates from a first encoded region, and wherein from a second vantage point decoded image light forming the portion originates from a second encoded region, different than the first encoded region.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationNo. 62/305,467, filed Mar. 8, 2016, the entirety of which is herebyincorporated herein by reference.

BACKGROUND

Various optical systems have been developed to enable object imaging,display output, and/or other functions. An optical system may beconfigured for non-contact object imaging using a lens array, forexample. As another example, a lens array may be provided with a displayto enable the output of floating images.

SUMMARY

Examples are disclosed that relate to optical systems. One exampleprovides a display device comprising an image source including aplurality of encoded regions from which encoded image light is output,and a Fourier transform array. The Fourier transform array may bepositioned to receive the encoded image light and output decoded imagelight that forms a floating image viewable from a plurality of differentvantage points, wherein from a first vantage point decoded image lightforming a portion of the floating image originates from a first encodedregion, and wherein from a second vantage point decoded image lightforming the portion originates from a second encoded region, differentthan the first encoded region.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter. Furthermore,the claimed subject matter is not limited to implementations that solveany or all disadvantages noted in any part of this disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view of a lens system having two lens arrayassemblies with cells that exhibit Fourier transform lenses inaccordance with one example.

FIG. 2 is a schematic, side view of the lens system of FIG. 1 inaccordance with an example having a respective in-tandem pair ofmicrolens arrays (MLAs) for each lens array assembly.

FIG. 3 is a schematic, side view of a MLA-based lens system inaccordance with another example.

FIG. 4 is a ray trace diagram for a lens system having two lens arrayassemblies in accordance with one example.

FIG. 5 is a ray trace diagram for a lens system in accordance with anexample in which each lens array assembly includes a respective array ofgraded-index microlensing structures.

FIG. 6 shows an example display device including a Fourier transformarray.

FIG. 7 shows an example display device including three displays whoseoutputs are optically combined using a lens array.

FIG. 8 shows an example display device including three displays whoseoutputs are optically combined using array-based imaging.

FIG. 9 shows an example display device configured to produce a floatingimage having a positive apparent z-distance.

FIG. 10 shows an example display device configured to produce a floatingimage having an apparent z-distance that is coplanar with an imagesource.

FIG. 11 shows an example display device configured to produce a floatingimage having a negative apparent z-distance.

FIG. 12 shows an example image.

FIG. 13 shows an encoded image corresponding to the image of FIG. 12.

FIG. 14 shows a magnified portion of the encoded image of FIG. 13.

FIG. 15 shows an example display device configured to produce a highresolution floating image.

FIG. 16 depicts an example of telecentric correction via an inner lensarray.

FIG. 17 shows an example display device including a GRIN lens arraystructure positioned to receive encoded image light.

FIG. 18 shows an example display device including an FT array using anin-tandem pair of microlens arrays positioned to receive encoded imagelight.

FIG. 19 shows an example display device in which light is opticallyencoded.

FIG. 20 shows a block diagram of an example computing device.

DETAILED DESCRIPTION

Relayed imaging involves the transfer of light of an object such as asource, mask, or sample to a photodetector array, substrate, or otherimage sensor or light-sensitive medium. Optical relay systems, such asarrays of graded index (GRIN) rod lenses, have been used in printers andfacsimile machines. The rods have a certain length to allow theirparabolic refractive index profile to image a given conjugate distance.The resulting systems are accordingly often too bulky for manyapplications. For example, the total conjugate length (the total tracklength of object plane to image plane) is often in the range of 9millimeters (mm) to 80 mm.

Lens systems provide relayed imaging via stacks or assemblies of lensarrays, such as microlens arrays (MLAs). The MLAs of the stacks areassembled such that conelets of light are stitched into a completenumerical aperture (NA) without gapping. Erect, high efficiency imagingis achieved. The lens system includes an imaging core of two lens arrayassemblies. In some cases, each assembly includes a pair of in-tandemMLAs. The MLAs of each pair are, in turn, separated by the focal lengthof the constituent lenslets (e.g., array elements) of the MLAs. The MLAsmay thus be disposed in a compact stack configuration. For example, oneimaging relay lens system has a total track length of 4.3 mm. The lenssystems are accordingly capable of imaging large areas while avoidingthe large volumes and bulky limitations of classical lens systems. Thelens systems also provide the relayed imaging with fewer parts thanother imaging relays.

The focal length separation of the two in-tandem pairs leads to highefficiency imaging. Each in-tandem pair implements a full, true Fouriertransform between position space and angle space (or spatial frequencyspace). A transformation into angle space is provided by the first pairat an intermediate transform plane between the two pairs. The secondpair then implements a second transformation from angle space back toposition space to provide the erect image at an image distancedetermined as a function of (i) the focal lengths of the MLAs, (ii) thepitches of the MLAs, and (iii) the distance between the two pairs, for agiven object distance. The function may thus be used to configure thelens system arrangement to form the image at a desired image distance.As described herein, formation of a real, non-inverted image is providedby satisfying the following two constraints: (1) providing consistentimaging conjugate distances within an array element, and (2) convergenceof image content across multiple imaging elements within the array.

Implementing a full Fourier transform avoids losses and otherdistortions by addressing the diffraction arising in connection witheach MLA. Without the second MLA for each cell, the transform appears tobe a Fourier Transform (FT) in intensity, but not in phase, as aquadratic phase error remains Phase correction is provided by the secondMLA of each pair, which effectively provides telecentric correction ofthe output. The inner array of each in-tandem FT MLA pair prevents lossand scatter of light having higher spatial frequency content, or lighthaving higher angle of incidence, at the intermediate transform plane.Without phase correction at these inner arrays, a portion of lightinvolved in fully resolving the object (within the diffraction limitdefined by lenslet acceptance numerical aperture NA) would be lost. Bymaking use of in-tandem FT MLA pairs, apodization is accordinglyavoided, thereby reducing diffractive artifacts and minimizing loss ofsystem resolve or loss of optical performance, such as ModulationTransfer Function (MTF). The fully formed diffraction output of thefirst in-tandem pair is then fully utilized by the second in-tandem pairto instead improve the efficiency of the lens system and, thus, imagequality. Clipping and vignetting are also avoided via the phasecorrection. The MLA-based lens system is instead capable of stitchingconelets of light into a complete numerical aperture (NA) without gaps.

Higher efficiencies may also be attainted due to a higher fill factor ofthe MLAs. Some microlens arrays are capable of being molded with 100%fill factor. For example, microlens arrays formed by replication usingan etched master or diamond machining may achieve 100% fill factor,while microlens arrays formed by photoresist reflow will have flat gapsbetween lenslets, and microlens arrays formed by grey-scale lithographymay exhibit rounded seams at the edges of each lenslet causing scatter.Other solutions, such as GRIN rod arrays, have a 10% loss in fill factorarising from hexagonal packing of round rods. Further, arrays of roundlenses have similar gaps in fill factor. By making use of high fillfactor types of MLAs or GRIN fiber faceplates from boules, or arrays oflenses each having a shape suitable for tiling, including hexagonal,rectangular, and square shapes, high efficiency may be maintained. MLAsmay utilize aperture array masking at any plane within the stack to helpreduce scatter of input light higher than the acceptance of eachlenslet, while extra mural absorption (EMA) glass or fiber may beutilized periodically within a GRIN fiber array to reduce such scatter.

The arrangement of the MLA pairs provides output without a tilingeffect, the so-called lensitization arising from lens system acceptanceand pointing angle. The tiling effect is avoided without having toresort to extensive increases in overall system length. The lens systemsmay therefore be provided in a compact, or thin, form factor (e.g., astack of sheets or thin films) appropriate for electronic devices, suchas phones, tablets, laptops, and other devices having a thin profile forportability and/or other reasons.

The relay imaging of the lens system is capable of being achieved in anon-contact manner. For example, a lens or other optical element is notnecessary at the image plane. Such non-contact imaging is useful whenimaging through transparent media, e.g., transfers through windows, asin the case of, for instance, fingerprint readers. The lack of contactis also useful in connection with transfers to substrates, as in thecase of, for instance, photolithography or transfers to an image sensorsuch as a microscope.

The lens systems are scalable without increases in system volume. Thelens systems are capable of being scaled to handle larger lateral areaswithout any increase in track length. The thickness of the lens systemthus does not increase. The lens systems may also be scaled toaccommodate different object conjugate distances without significantincreases in volume. Such scaling may also be useful in connection withimaging at short conjugate lengths. For instance, total conjugatelengths less than 9 mm are achievable.

The lens systems are telecentric in some cases. For example, the twoin-tandem MLA pairs may be arranged to provide telecentric output forobjects that are sufficiently far enough away from the lens system.Telecentric output is achieved without having to rely on a physicallayer (e.g., a field lens) at the image plane. The lens systemsaccordingly avoid introducing mechanical interference issues inconnection with, for instance, photolithography (e.g., avoiding contactwith the wafer being exposed). In other cases, field correction may beused to achieve symmetric behavior. Input and output may besubstantially telecentric. The ability to provide telecentric outputavoids distortion and defocusing present in previous lens systems.

The two in-tandem MLA pairs of the lens systems may be arranged toprovide unity or non-unity magnification. The respective focal lengthsor respective pitches of the MLA pairs differ in non-unity magnificationexamples.

In some cases, the relay imaging of the lens systems is provided inconnection with digital integral imaging.

Although described in connection with digital integral imaging inconnection with portable electronic devices (e.g., tablets havingtouchscreens), the lens systems are well suited for a wide variety ofdigital integral imaging applications and usage scenarios.

The lens systems are not limited to particular types of MLA-based arrayassemblies. Other types of lensing structures and arrays may be used foreach one of the lens array assemblies. For instance, each lens arrayassembly may include an array of GRIN microlensing structures. Each GRINmicrolensing structure of the array then corresponds with a respectiveone of the cells of each lens array assembly. As used herein, the term“cell” is used to refer to a unit of each array assembly. Because, insome cases, an array assembly includes a pair of arrays, a cell mayinclude an FT pair of array elements, one from each array. In othercases (e.g., GRIN cases), a cell corresponds with a single element ofthe array that provides the equivalent of an FT lensing pair.

FIG. 1 is a schematic view of a lens system 100 configured to operate asan imaging relay. The lens system 100 includes two lens arrayassemblies. In this example, one assembly of the lens array assembliesof the lens system 100 includes a first pair 102 of in-tandem microlensarrays 104. The other assembly of the lens array assemblies 100 includesa second pair 106 of in-tandem microlens arrays 108. Each lens arrayassembly has a plurality of cells. As described below, each cell isconfigured to exhibit a pair of Fourier transform lenses. In thisexample, each microlens array 104, 108 includes a respective set ofconstituent lenslets 110, respective pairs of which make up each cell ofthe lens array assemblies.

Light from an object 112 diffuses as it approaches the lens system 100.The object 112 is separated from the lens system 100 by an objectdistance z₁. A few example rays of light are shown in FIG. 1. Theexample rays propagate from a point A on the object 112 toward the firstpair 102 of arrays 104. In many cases, light from the object 112encounters each of the lenslets 110 to the extent permitted by thenumerical aperture, or acceptance cone, of the microlens array 104.

The microlens arrays 104, 108 of the two array pairs 102, 106 arepositioned to achieve relay imaging. Each array 104, 108 is generallydisposed, or oriented, along a respective plane, as shown in FIG. 1. Therespective planes and, thus, the arrays 104, 108, are separated from oneanother along an optical axis 114. The arrays 104 of the first pair 102are spaced from one another by a distance f₁. The arrays 108 of thesecond pair 106 are spaced from one another by a distance f₂. The arraypairs 102, 106 are spaced from one another by a distance t_(g) (or D).Each distance is an effective optical distance determined in accordancewith the refractive index of the medium through which light passes whentransmitted over the particular distance. Each distance is selected inaccordance with a function that establishes the image conjugate distancefor the relay imaging of the lens system 100. The image conjugatedistance is established by satisfying the constraints of (1) providingimaging conjugate distances within a lenslet as well as (2) convergenceof image content across multiple imaging cells within the assembly,thereby enabling formation of a real, non-inverted image.

The distances f₁ and f₂ are set in accordance with the focal lengths forthe cells, e.g., the lenslets 110 of the arrays 104, 108. The distancef₁ is the common focal length of the cells of the first lens arrayassembly, e.g., the lenslets 110 of the arrays 104. The distance f₂ isthe common focal length of the cells of the second lens array assembly,e.g., the lenslets 110 of the arrays 108.

The focal length separation of each array 104 of the pair 102establishes that the array pair 102 implements a Fourier transform ofthe light emanating from the object 112. For objects at an infinitedistance from the first array pair 102, t_(g) is zero and the Fouriertransform is a phase-corrected Fourier transform from the angle space ofthe light emanating from the object 112 into position space (or spatialfrequency space), as explained herein. The array pair 102 provides arepresentation of the phase-corrected Fourier transform along a plane116 disposed between the array pairs 102, 106. The plane 116 isaccordingly referenced as an intermediate transform plane. For closerobject distances, the distance t_(g) increases, such that theintermediate transform plane 116 exists at a finite distance from andbetween the arrays 104 and 108. As described below, for a given lensletdesign, the distance or optical gap t_(g) follows a mathematicalrelationship dependent on object distance along with other lensletparameters. A stack having a fixed t_(g) may function reasonably wellover a limited range of object distances in proximity to the designobject distance.

Use of two-lens in-tandem Fourier transform MLA pairs enables higherspatial frequency content (corresponding to higher angle light) totransmit into the intermediate transform plane without clipping. Suchtransmission, in turn, allows formation of a Sinc-like function that ismore highly resolved, containing higher spatial frequency content, andlimited primarily only by MLA acceptance numerical aperture (NA). Thisin turn allows the converging conelets out of each cell to be stitchedforming a core NA without gaps within the solid angle of the NA. Impacton the Fourier transform due to lenslet sag profile may be reduced byusing aspheric lenslet profiles, such as a conic constant in the rangeof −0.25 to −0.4, or other aspheric profiles.

The focal length separation of each array 108 of the pair 106establishes that the array pair 106 implements a Fourier transform ofthe light associated with the representation at the intermediatetransform plane 116. The Fourier transform is again a phase-correctedtransform. The array pair 106 transforms the representation at theintermediate transform plane 116 from angle space back into positionspace.

The two array pairs 102, 106 are positioned relative to one anotheralong the optical axis 114 to establish that the lens system 100 is animaging system. That is, the distance D between the two array pairs 102,106 establishes that the lens system 100 provides an erect image 118 ofthe object 112. The image 118 is provided at an image conjugate distancez₂ from the array pair 108.

The image conjugate distance z₂ is established via a function of theobject conjugate distance z₁ for the object 112, the distance D betweenthe array pairs 102, 106, a first pitch of the first array pair 102, asecond pitch of the second array pair 106, and the common focal lengthsf₁, f₂. The function establishes that the light emanating from theobject 112 and passing through the constituent lenslets of the in-tandemmicrolens arrays of the array pairs 102, 106 converges at the imageconjugate distance z₂. Further details regarding the function areprovided hereinbelow in connection with parameters identified in FIG. 1.

For two lenses in tandem, f_(1a) and f_(1b), separated by distance D,the distance s_(i) after the last lens at which an image of the inputobject 112, at distance z₀ before the first lens, occurs may be definedas

$s_{i} = \frac{f_{1b}\left( {{F\left( {f_{1a} - z_{o}} \right)} + {f_{1a}z_{o}}} \right)}{{D\left( {f_{1a} - z_{o}} \right)} + {f_{1b}z_{o}} + {f_{1a}\left( {z_{o} - f_{1b}} \right)}}$

However, when focal lengths f_(1a) and f_(1b) are configured as aFourier Transform pair, such that f_(1a)=f_(1b)=f₁ and separationdistance D=f_(1a)=f₁, then the image distance of input object A, occursat distance s_(i) after the last lens, which simplifies to z_(g1):

$z_{g\; 1} = {\frac{f_{1}\left( {{f_{1}\left( {f_{1} - z_{o}} \right)} + {f_{1}z_{o}}} \right)}{{f_{1}\left( {f_{1} - z_{o}} \right)} + {f_{1}z_{o}} + {f_{1}\left( {z_{o} - f_{1}} \right)}} = \frac{f_{1}^{2}}{z_{o}}}$

where f₁ is the focal length of each lens of the two-lens in-tandemFourier transform pair and z₀ is the object distance before the firstlens.

The foregoing relationship may then be extended to the array context. Anarray of lenslets, or cells, are formed by pitch d. A portion of lightdiverging from the object 112 is captured by each cell. Each cell in onearray forms a two-lenslet subsystem with a cell in the other array of anarray pair. For a solid angle of light from the object 112 thatoverfills a cell of width near pitch d, the input captured isapproximately a Rect function which forms a Sinc-like function near theimage of A at the intermediate transform plane defined by, or disposedat, the distance z_(g1) from the second array in the array pair.

The second Fourier transform array pair 106 is placed after the firstarray pair 102 at gap distance optically equivalent to t_(g)=2*z_(g1).The configuration thus becomes symmetric. The imaging conjugatedistances provided by each subsystem are the same. The images developedby all of the subsystems converge for image formation of the object 112,at distance z_(i), to form image 118 (see, e.g., point A′ correspondingto point A on the object 112). In such cases, the imaging relay becomesa 1:1 relay such that the image distance z_(i) is substantially equal tothe object distance z₀.

The intermediate images may be referred to as intermediate transformimages of the input object 112, which occur at the intermediatetransform plane near half the gap t_(g), defined previously as distancez_(g1).

The distance, or gap t_(g), between the two array pairs 102, 106 isdeterminative of the imaging of a stack of cells. The cell stackincludes four cells, one from each array 104, 108. Each cell stack maybe considered a constituent sub-system of the lens system 100. Thedistance between the two array pairs 102, 106 is selected such thatimaging is achieved for all rays entering the constituent sub-system ata common image conjugate distance. The distance, or gap t_(g), increasesas the object distance decreases (i.e., the object 112 becoming closerto the lens system 100). The distance, or gap t_(g), goes to zero as theobject distance goes to infinity (or very large distances relative tothe dimensions of the lens system 100). In the example of FIG. 1, thetwo array pairs 102, 106 are spaced apart from one another by a gap 120.The distance, or gap t_(g), for the function is thus non-zero. The gap120 may be on an order of, or in the proximity of, the first and secondcommon focal lengths.

The pitch of the lenslets 110 within the arrays 104, 108 governs theconvergence of light from all of the lenslets 110. The lenslets 110 ofthe first array pair 102 have a pitch d₁, while the lenslets 110 of thesecond array pair 106 have a pitch d₂. The pitch is selected such thatconvergence of all optical information across all lenslets 110 of thearrays 104, 108 is achieved. An image is thus formed at the same imageconjugate distance across all lenslets 110 of the array 104, 108. Thelens system 100 is an example of an imaging relay in which the lenslets110 of both array pairs 102, 106 have a common pitch. With the pitchesd₁, d₂ equal to one another, the output of the lens system 100 may betelecentric.

Telecentric output may also be provided, on one side of the opticalstack, in non-equal pitch cases, i.e., when d₁ does not equal d₂. Insuch cases, the pitches of the lenslets 110 in each array 104 of thefirst array pair 102 are equal to one another, and the pitches of thelenslets 110 in each array 108 of the second array pair 106 are equal toone another. The function simplifies as follows:

$d_{2} = \frac{d_{1}{z_{2}\left( {f_{1} + z_{1}} \right)}}{\left( {f_{2} + z_{2}} \right)z_{1}}$

The gap t_(g) is as follows:

${t_{g} = {z_{g\; 1} + z_{g\; 2}}},{{{where}\mspace{14mu} z_{g\; 1}} = {{\frac{f_{1}^{2}}{z_{1}}\mspace{14mu} {and}\mspace{14mu} z_{g\; 2}} = {\frac{f_{2}^{2}}{z_{2}}.}}}$

In such case, the pitches are configured such that d₁=d_(1b)<d_(2b)=d₂.

Non-telecentric imaging, on both sides of the optical stack, may also beprovided. The rays may be smoothly bent through the lens system 100 byadjusting the respective pitches of the cells within the arrays 104,108. The lenslets 110 of the arrays 104, 108 may thus be registered (oraligned) with one another or non-registered. In one example, the pitchesof all four arrays differ from one another. The pitch d₁ for thelenslets 110 becomes d_(1a) and d_(1b) for the first and second arrays104 of the first array pair 102. The pitch d₂ for the lenslets 110becomes d_(2a) and d_(2b) for the first and second arrays 108 of thesecond array pair 106. In one positive magnification case,d_(2b)>d_(2a)>d_(1b)>d_(1a). The function then may be expressed asfollows (with z_(g1) and z_(g2) defined as set forth above):

$d_{2} = \frac{{d_{2b}f_{2}z_{1}} + {d_{1b}f_{1}z_{2}} + {d_{1}z_{1}z_{2}}}{z_{1}\left( {{2f_{2}} + z_{2}} \right)}$

In such case, the pitches are configured such that d₁<d_(1b)<d_(2b)<d₂.

As shown by the examples described above, the gap t_(g) is determinativeof the imaging of each subsystem of cells, while the relative pitches ofthe arrays govern the convergence from all the cell subsystems.

The focal lengths of the cells within the arrays 104, 108 may also beused to adjust the image conjugate distance. Non-unity conjugatedistances may be achieved when the focal lengths of the lenslets 110within the arrays 104 are not equal to the focal lengths of the lenslets110 within the arrays 108. In the example of FIG. 1, the focal lengthsof the lenslets 110 within the arrays 104 and 108 are equal to oneanother.

The term “equal” is used herein to mean exactly equal and effectivelyequal. Effectively equal may include, for instance, parameters that areequal within a reasonable margin of error, such as a manufacturingtolerance. The parameters values thus need not be exactly equal (e.g.,slightly offset) to be considered “equal” as that term is used herein.Any of the parameters described herein as equal in some examples mayalternatively be “substantially equal” in other cases. Substantiallyequal parameter values may be intentionally or unintentionally offset bya slight amount that results in a discernable (e.g., detectable), butinsignificant, effect on system output. Any of the parameters describedherein as equal in some examples may alternatively be “about equal” inother cases. About equal parameter values may be intentionally orunintentionally offset by a slight amount that results in a discernable(e.g., detectable) effect on system output that may be consideredsignificant in some applications but insignificant in otherapplications. For example, a slight de-focusing of system outputresulting from about equal parameters may be significant in the contextof a fingerprint reader, a microscope, or photolithography, butinsignificant in the context of a printer or facsimile machine.

Distances referenced herein, such as the width of the gap 120, maydiffer in practice in accordance with the refractive index of thetransmission medium. For example, the above-described functions specifya distance for the gap parameter in connection with transmission throughan air gap. The actual width of the gap 120 may differ from the air gapdistance if the light is propagating through a medium other than airwhen traversing the gap 120. The gaps and other distances may thus beoptically equivalent distances. In cases using an optical medium otherthan air, the inner lenslet focal lengths may be adjusted to account forchange in curvature required to maintain the Fourier Transform functionof each pair. Increase in refractive index in the gap implies smallerlenslet curvature to maintain substantially equal effective focal lengthfor an in-tandem MLA pair. Further, such practice is useful when it isdesired to laminate both MLA pairs to form an optical stack thatincludes a monolithic optical stack without an air gap.

Optical terms such as “collimated”, “focused”, etc., are used herein toinclude both the exact condition described by the term as well asconditions near the exact condition. For example, light is considered tobe collimated if the light rays are collimated to an effective extentfor purposes of the imaging application or usage scenario involved. Theresolution of the viewer may thus be taken into account when evaluatingwhether the optical condition (e.g., collimated, focused) is present.

FIG. 2 depicts a side view of an MLA-based lens system 200 in accordancewith one example. As in the examples described above, the lens system200 includes two array pairs 202, 204. The array pair 202 includesarrays 206, and the array pair 204 includes arrays 208. In this example,each array 206, 208 is disposed on a respective substrate 210. Thesubstrates 210 may or may not be similarly configured and constructed.In one example, each substrate 210 is composed of a glass substratehaving a thin microlens layer replicated on one surface using UV-cureadhesive resin, which may be cured using ultraviolet light, and a moldmaster, and each substrate 210 may have a similar thickness.

Each array 206, 208 includes a set of lenslets 212. In one example, eachsubstrate 210 and set of lenslets 212 is integrally formed via injectionmolding. Alternatively, the lenslets 212 may be formed separately fromthe substrate 210 and affixed or otherwise secured thereto. For example,the lenslets 212 may be formed, and then applied to the substrates 210with optically clear adhesive. The pitch and focal length of thelenslets 212 may be equal or offset as described above. Other techniquesfor forming the lenslets 212 may be used.

Surfaces may be optically coated with anti-reflective (AR) coatings tominimize loss and scatter due to Fresnel reflections. In cases usingUV-cure adhesives, a glass master may be utilized to minimizedistortion. Other examples include injection molding in a homogenouspolymer, such as acrylic or polycarbonate, compression molding ofpolymer sheets, and nanoprinting. For compression molding, a nickel shimmay be formed of a master negative using an electroform nickel process.A master may also be formed using diamond machining, such as plungecutting a rotating cutter having a profile form for each lenslet, ordiamond turning each lenslet. For high accuracy and fill factor, alithography based glass etching technique may be used to fabricate theMLA master.

Various profiles may be used for each lenslet 212. For example, thelenslets 212 may have aspheric or conic profiles. The extent or degreeto which the profiles are aspheric or conic may vary. In some cases, thelenslets 212 may have profiles with conic constants directed to reducingaberrations and/or allow the lens system to accommodate higher numericalapertures (NA).

The arrays 206, 208 are oriented symmetrically about the intermediatetransform plane (FIG. 1). Each lenslet 212 of the arrays 206, 208 is aplano-convex structure. The planar side of each structure is adjacentthe respective substrate 210. The curved side of each structure isdisposed on an inward facing side of each array 206, 208 in the exampleof FIG. 2. The orientation of the lenslets 212 may vary from the exampleshown. One example is described below in connection with FIG. 3.

The lenslets 212 may be arranged in a variety of two-dimensionalpatterns. For example, the lenslets 212 may be disposed in a hexagonalarray, a square array, or other arrangement. The lateral shape of eachlenslet 212 may vary accordingly.

One or more of the arrays 206, 208 may be pattern or aperture masked.The aperture masking may be directed to limiting the acceptance of thelens system and/or reducing scattered light. Pattern masking may bedirected to blocking ambient or other spurious light from beingprocessed by the lens system 200. For instance, ambient light at highangles of incidence upon the lens system 200 is blocked. Blocking thehigh angle incident light may prevent the spurious light from hopping toa neighboring subsystem of cells. In the example of FIG. 2, the lenssystem 200 includes aperture stops 214 along the inner array 206 of thefirst array pair 202, as well as aperture stops 216 along the innerarray 208 of the second array pair 204. Fewer, additional, oralternative aperture stops 214, 216 may be provided. For example, otherlens systems may not include the aperture stops 216. Aperture stops maybe disposed at any one or more of the layers.

The aperture stops may be provided for other purposes. For example, theaperture stops may address aberrations in the lenslets of the arrays.

Aperture masking may be accomplished by using a lithographed aperturehole array layer, such as patterned deposited thin metal oxide or metal,on top of or underneath the replicated layer as by replicating over anaperture-patterned substrate surface, or one or more patterned sheetsdisposed within the optical stack, e.g., near the inner arrays.

The aperture stops 214, 216 may be provided via a discrete sheet orother layer disposed along the respective array 206, 208. For example,an opaque sheet secured to, or disposed along, the array 206 may includean array of apertures to define the aperture stops 214. Alternatively oradditionally, the pattern masking may be provided via a layer depositedor otherwise formed on the respective substrate 210. The layer may bepatterned to define the aperture stops 214, 216 before the formation ofthe lenslets 212.

The aperture stops 214, 216 may be embedded in, or otherwise integratedwith, the lenslets 212. For instance, the material of each aperture stop214, 216 may be deposited on the substrate 210 and then patterned beforeand/or in connection with the formation of the lenslets 212. Further, aperforated or ‘honeycomb’ sheet of limited thickness may be disposedbetween the pairs.

FIG. 3 depicts another example lens system 300 having array pairs 302,304. In this case, arrays 306 of the pair 302 are disposed on oppositesides of a substrate 308. Arrays 310 of the pair 304 are disposed onopposite sides of another substrate 312. Each array 306, 310 includeslenslets 314, each of which includes a plano-convex structure. Thecurved surface of each lenslet 314 faces inward or outward in accordancewith the side of the substrate 308, 312 on which the lenslet 314 isdisposed.

The lenslets 314 may be arranged, shaped, formed, and otherwiseconfigured as described above. The lens system 300 may have alternativeor additional aspects in common with the examples described above. Forexample, in some cases, the lens system 300 includes aperture stops onone or more of the arrays 306, 310.

The arrays of the examples of FIGS. 2 and 3 may be separated from oneanother by air. Other media may be used. For example, low refractiveindex adhesive or laminate materials may be disposed between the arrays.

Lenslet orientations other than those shown in FIGS. 2 and 3 may beused.

The lens systems may differ from the examples of FIGS. 2 and 3 in otherways. For example, the arrays of each pair may be in contact with oneanother. In four-substrate cases, the arrays may be in contact as aresult of the thickness of the inner substrates.

Additional substrates or other structures may be used in other examples.For example, two substrates may be disposed before and after the arrayassemblies of FIG. 3 to provide, for instance, additional structuralsupport or protection.

A four layer MLA stack may be configured to accommodate conjugatedistances from infinity to close proximity. However, the gap distancet_(g) may increase substantially for object distances approaching thefocal length of the first array. In such cases, additional, outer MLAsmay be added in order to enable the inner gap distance to besubstantially reduced, and further provide a focus NA having no gaps.Further, the outer MLAs may provide added freedom in design as eachsurface profile may be tailored to improve optical performance.

FIG. 4 is a ray tracing diagram to provide a paraxial illustration for afour-layer lens system 400 that may be used as, for instance, an imagingrelay 400. The pitch, focal length, and conjugate distance parameters ofthe lens system may be selected in accordance with the functiondescribed above to achieve, for instance, 1:1 relay imaging.

FIG. 5 shows a GRIN-based lens system 500 in which two lens arrayassemblies 502, 504 are positioned relative to one another for relayimaging. Each assembly 502, 504 includes an array of GRIN microlensingstructures 506.

Each structure 506 provides one of the cells of the lens array assembly502, 504. To that end, each structure 506 is configured to exhibit, oract as, a pair of Fourier transform lenses. A parabolic refractive indexprofile enables each cell to function as a series of two in-tandemFourier transform pairs, in order to satisfy the above-describedconstraints of (1) image conjugate formation for each cell and (2)convergence of imaging from multiple cells. Each assembly 502, 504 maythus be configured as a plate having a thickness that provides theequivalent function of the above-described Fourier transform cellsformed via a lenslet pair.

EMA glass may be used to block light from encountering the edge of anadjacent cell. Other absorbing coatings or layers may be used.

For given total track length (TTL) between image conjugates, the objectand image distances may be placed by design to coincide with the outerfaces of the GRIN lens length, thus having image conjugates at the outersurfaces, or may be placed at an air gap distance from each of the twoouter surfaces. However, the structures 506 are unlike previous GRINarrays used for image relay or image transfer that have been based ondesigning a single GRIN lens length, thus only functioning for aspecific pair of conjugate distances. By realizing herein that afundamental element required in enabling the formation of images inarray-based imaging is a series of two Fourier Transform (FT) equivalentsub-systems within each cell of the array, and the Fourier Transformequivalent length for a GRIN lens may be defined as the length for whicha collimated input beam forms a focus at the second output surface ofthe GRIN lens, an air gap (zero to non-zero) defined by image conjugaterelations may be disposed between the two Fourier Transform equivalentlength GRIN arrays in order to provide array-based relay imaging, whichmay add versatility of the system to be adjusted for any pair of equalimage conjugate distances using two FT-equivalent length GRIN arrayswhich have the same effective focal length, and further may be adjustedto support unequal conjugate distances by use of two FT equivalentlength GRIN arrays which have different focal length. Such arrays may bemade thinner by use of GRIN fibers or drawing an array of GRIN rods intoa boule having accurate placement of array spacing, then slicing andpolishing faces, in a similar fashion as coherent fiber optic faceplatesare fabricated, but with tight control of pitch layout. Further, while aGRIN lens is a lens which exhibits a continuous lensing effect oroptical power anywhere along its length, a Fourier Transform equivalentsubsystem may also be formed using two or more lens arrays. While asingle lens array may be used to form an array of output images whichappear to be Fourier Transforms in terms of intensity, these are notFourier transforms in terms of phase, or telecentrically corrected uponoutput. The simplest form of an FT equivalent subsystem would theninclude two lenses having the same focal length placed in tandem at aseparation distance equal to the effective focal length. However, it isclear from the GRIN lens FT equivalent length case explained above, thatmore than two lenses having various optical power may be used in seriesto achieve the equivalent function of a FT-equivalent subsystem. Thesimplest case of using microlens arrays to achieve array-based imaginginvolves use of two in-tandem Fourier Transform pairs of microlenses,having an air gap or optical path length gap distance defined by imageconjugate relations which will be defined below. In such way, any lensarray, including microlens arrays, GRIN lens arrays, or GRIN microlensarrays may be utilized to form an imaging relay, as explained below.

FIG. 6 schematically shows an example display device 600 that includesan image source 602 from which encoded image light is output andtransmitted through an FT array 604 to thereby form a floating imageviewable from a plurality of different vantage points. FIG. 6 depicts,for a first encoded region 606A of a plurality of encoded regions 606 ofimage source 602, encoded image light 608A output from the first encodedregion and received by FT array 604, which is positioned to receiveencoded image light from the plurality of encoded regions. Encoded imagelight 608A is then transmitted through the FT array, which therebydecodes the encoded image light and outputs decoded image light 610A.Decoded image light 610A forms a portion 617A of a floating imagefocused at a focal plane 612. Decoded image light from other encodedregion(s) of the plurality of encoded regions 606 may combine withdecoded image light 610A to form a complete floating image. The floatingimage is viewable from a first vantage point 614, at which a viewer'seye is represented. While the example depicted in FIG. 6 shows two lightrays forming the floating image as viewed from first vantage point 614,any suitable number of light rays may form the floating image from thefirst vantage point. In some examples, FT array 604 may output asubstantially continuous field of light rays to first vantage point 614.Details regarding the encoding of light output by image source 602, andthe decoding of encoded image light performed by FT array 602, aredescribed below with reference to FIGS. 12-14.

The floating image is further viewable from a plurality of other vantagepoints in addition to first vantage point 614. As an example, FIG. 6shows a second vantage point 616 from which the floating image may beviewed. From the second vantage point 616, decoded image light 610Boriginates from a second encoded region 606B from which encoded imagelight 608B is output. The portion 617A of the floating image formed bydecoded image light 610B (when viewed from vantage point 616) is thesame portion 617A of the floating image formed by decoded image light610A (when viewed from vantage point 614). In other words, differentregions of the image source output light that makes up the same portion617A of the floating image, and which of these different regionssupplies light to the eye depends on the vantage point from which theeye views the floating image.

As can be seen from the example depicted in FIG. 6, decoded image lightforming the floating image at focal plane 612 may originate fromdifferent encoded regions of the plurality of encoded regions 606 fordifferent vantage points from which the floating image is viewed. Asdescribed above, the portion 617A of the floating image as viewed fromfirst vantage point 614 is formed by decoded image light 610Aoriginating from encoded region 606A, which differs from encoded region606B from which decoded image light 610B originates and forms theportion 617A of the floating image as viewed from second vantage point616. As the floating image is viewable from a plurality of vantagepoints (e.g., within the field-of-view of FT array 604) includingvantage point(s) other than first and second vantage points 614 and 616,decoded image light forming the floating image may originate fromdifferent sets of encoded regions of the plurality of encoded regions606 for different vantage points, respectively.

Image source 602 may assume a variety of suitable forms. As one example,image source 602 may include a display image output from a display. Anysuitable type of display may be used to provide the display image,including but not limited to a two-dimensional backlit liquid crystaldisplay (LCD) and organic light-emitting diode (OLED) display. In someexamples, the display may be operated to output a sequence of displayimages to thereby provide animated image content (e.g., video). In otherexamples, image source 602 may include a static image provided by a(e.g., encoded) film or other light emitting/transmitting/reflectingsurface.

FT array 604 may implement a Fourier transform of encoded image lightoutput by image source 602 to thereby decode the encoded image light andprovide decoded image light forming the floating image at focal plane612. To this end, FT array 604 may assume various suitable forms. Forexample, FT array 604 may include at least a portion of one or more oflens systems 100, 200, 300, 400, and 500 of FIGS. 1-5, respectively.More specifically, FT array 604 may include a lens array pair such assecond pair 106 of lens system 100, array pair 208 of lens system 200,array pair 310 (e.g., with substrate 312) of lens system 300, and/orlens assembly 504 of lens system 500. Thus, FT array 604 may include alens array pair and/or a graded-index microlens structure, and furthermay implement a Fourier transform from position space to angle space.Encoded content accordingly may be provided by an image source such asimage source 602 to second pair 106 of lens system 100, array pair 208of lens system 200, array pair 310 (e.g., with substrate 312) of lenssystem 300, and/or lens assembly 504 of lens system 500.

Focal plane 612 may represent a reconvergent plane at which a 2Dfloating image is formed. Further, various portions of encoded contentwithin each encoded region 606 may converge at different apparent focalplanes, such that decoded light forms a converged 3D floating image.Further still, an array of phase delays may be disposed in between imagesource 602 and FT array 604, such as via use of a spatial lightmodulator (SLM), such that both intensity and phase of encoded contentwithin each encoded region may be controlled for cases using coherentillumination.

In some implementations, image source 602 may include a stitched displayimage formed by optically combining output from two or more displays. Tothis end, FIG. 7 shows an example display device 700 that includes threedisplays 702A, 702B, and 702C whose outputs are optically combined usinga lens array. In particular, a first lens array 704 is positioned toreceive image light from displays 702 and transmit the image light to asecond lens array 706 positioned to receive the image light from thefirst lens array. As examples, FIG. 7 shows a first region 708 at whichoutput from first and second displays 702A and 702B is stitched, oroptically tiled, and a second region 710 at which output from second andthird displays 702B and 702C is stitched. First and second regions 708and 710 may be substantially continuous regions in which a plurality oflight rays from multiple displays are combined. While the example shownin FIG. 7 depicts optical stitching and tiling of three displays, anysuitable integer number of displays equal to or greater than two may bestitched and tiled in this manner. Further, first lens array 704 may beconsidered as an array of projection lenses, and displays 702 may beconsidered analogous to or replaceable with an array of projectiondisplays. Still further, image source 602 may include image sourcesshowing static content, such as from a display displaying a fixed imageor as from lithographed, screened, or printed matter, or may includeimage sources which may be updated over time, such as time-sequentiallyupdated content displayed on an electronically addressable display, suchas video-rate content, or further still may include image sources havingone or more layers of electronically addressable displays, such stackhaving thickness, which may provide more than one conjugate z-distance.

First lens array 704 inverts the image light output from displays 702,which thus may output encoded image light that is inverted relative tothe image light that ultimately forms a viewable floating image. Secondlens array 706 performs telecentric correction of the image lightreceived from first lens array 704. To this end, second lens array 706may be a high fill-factor lens array configured to telecentricallycorrect pointing angle v. position, which may enable the formation of alarger, relatively higher resolution image source that can be employedas an input display object placed at an intermediate transform planeoptically upstream of the latter portion of an array-based imagingoptical stack. As one example with reference to FIG. 6, the higherresolution image source may be employed as image source 602, in whichcase stitched, encoded image light generated by display device 700 maybe transmitted to FT array 604 to thereby produce a stitched, decodedimage. Additional detail regarding telocentric correction in anin-tandem pair of lens arrays is described below with reference to FIG.16.

In some implementations, output from two or more displays may beoptically combined via array-based imaging. To this end, FIG. 8 shows anexample display device 800 that includes three displays 802A, 802B, and802C whose outputs are optically combined using array-based imaging. Inparticular, a reconvergence sheet 804 is positioned to receive imagelight from displays 802 and transmit the image light to a lens array 806positioned to receive the image light from the reconvergence sheet. Asexamples, FIG. 8 shows a first region 808 at which output from first andsecond displays 802A and 802B is stitched, and a second region 810 atwhich output from second and third displays 802B and 802C is stitched.First and second regions 808 and 810 may be substantially continuousregions in which a plurality of light rays from multiple displays arecombined. While the example shown in FIG. 8 depicts optical stitchingand tiling of three displays, any suitable integer number of displaysequal to or greater than may be stitched and tiled in this manner.

Reconvergence sheet 804 receives non-inverted image light output fromdisplays 802 and performs tiling of the output. Reconvergence sheet 804may include lens arrays of dissimilar pitch which define a repeatinggrid of image conjugate pairs for each display 802, such that eachdisplay is aligned at an object position, so as to tile display imagesseamlessly at the plane of lens array 806. Lens array 806 performstelecentric correction of the light output by reconvergence sheet 804 toenable the formation of a larger, relatively higher resolution imagesource that can be employed as an input display object placed at anintermediate transform plane optically upstream of the latter portion ofan array-based imaging optical stack. As one example with reference toFIG. 6, the higher resolution image source may be employed as imagesource 602, in which case stitched, encoded image light generated bydisplay device 800 may be transmitted to FT array 604 to thereby producea stitched, decoded image. In such case, the tiled display plane formedjust after lens array 806 may be used as image source 602.

One or more of the lens systems and display devices described above maybe configured to produce a floating image at a variety of apparentz-distances relative to an image source. FIG. 9 shows an example displaydevice 900 that is configured to produce a floating image having anapparent z-distance z_(i,+) that is positive relative to an image source902. As the light rays that form the floating image are focused at afocal plane 904 located at the apparent z-distance z_(i,+), the floatingimage appears to float in front of image source 902 as viewed from oneor more vantage points, such as vantage point 906 and/or vantage point908.

The floating image may be a two-dimensional (e.g., planar) orthree-dimensional image that appears to have depth. A 3D floating imagehas different image portions appearing at different z positions formultiple vantage points, thus having content forming a combination ofapparent z-distances. For simplicity, 3D images of the presentdisclosure may be referred to as having a z-distance, which may beadjusted forward or backward. In practice, when a 3D image is z-shiftedforward or backward, all the different z-distances of the differentcontent portions will shift accordingly.

A floating image may be produced at other locations relative to an imagesource. As another example, FIG. 10 shows an example display device 1000that is configured to produce a floating image having an apparentz-distance z_(i,c) that is coplanar with an image source 1002. As lightrays that form the floating image are focused at a focal plane 1004 thatis coplanar with the plane of image source 1002, the floating imageappears to originate from the image source as viewed from one or morevantage points such as vantage point 1006. The floating image may be atwo-dimensional (e.g., planar) or three-dimensional image that appearsto have depth, which may thus include decoded image light converging atone or more z-distances to produce image content within the floatingimage.

As yet another example of the potential placement of floating imagerelative to an image source, FIG. 11 shows an example display device1100 that is configured to produce a floating image having an apparentz-distance z_(i,−) that is negative relative to an image source 1102. Aslight rays that form the floating image appear to be focused at a focalplane 1104 located at apparent z-distance z_(i,−), the floating imageappears to be located behind image source 1102 as viewed from one ormore vantage points such as vantage point 1106. The floating image maybe a two-dimensional (e.g., planar) or three-dimensional image thatappears to have depth. Further, in view of the above, “floating image”as used herein may refer to an image that appears in front of, coplanarwith, or behind an image source.

A variety of approaches may be used to achieve a desired z-distance of afloating image relative to an image source. As described above, variousoptical parameters of a lens system may be selected to achieve a desiredfloating image z-distance, including but not limited to lens focallength, lens pitch, and/or the distance between adjacent lens arrays. Toachieve a negative floating image z-distance, one or more lens arraysmay be placed behind the outer surface of a display—e.g., between abacklight and a cover layer of the display. For emitting display types,such as OLED, a slightly reduced thickness of an FT-pair (with orwithout a slight modification of the focal length), may be used in orderto achieve desired imaging performance. As another example, a desiredfloating image z-distance may be achieved by selecting the pitch ofencoded regions (e.g., the space between adjacent encoded regions suchas encoded regions 606A and 606B, both of FIG. 6) relative to the lenspitch of one or more lens arrays. As yet another example, an actuatormay be used to dynamically vary the position of one or more lens arraysrelative to an image source. Returning briefly to FIG. 6, the potentialinclusion of an actuator 618 operatively coupled to FT array 604 isshown. Actuator 618 may be configured to adjust a position of FT array604 relative to image source 602 to thereby adjust an apparentz-distance of the floating image produced by display device 600.Actuator 618 may assume any suitable form, such as that of apiezoelectric device or nanomotor. Still further, for examples in whichan image source includes a display image provided by a display, thedisplay may be driven to vary its output (e.g., encoded content) tothereby adjust the focal convergence range of decoded ray bundles toadjust an apparent z-distance of a floating image. By adjusting both theposition of FT array 604 relative to image source 602, as well as makingcorresponding adjustments to the encoded content within encoded regions,a floating image may form which exhibits convergence both optically anddigitally. It will be appreciated that one or more of the mechanismsdescribed above may be employed to vary the apparent z-distance of afloating image within a range of z-distances, and/or to switch theapparent z-distance to one or more of a z-distance that appears in frontof, coplanar, and behind an image source.

In some implementations, one or more of the display devices describedherein may cooperate with an image sensor to track the vantage point ofa user and adjust operation in response to changes in the vantage point.For example, a visible light, infrared, and/or depth camera may be usedto determine the position of one or both of a viewer's eyes relative toa display, which may be used to adjust the encoded content provided bythe display. The encoded content may be adjusted to maintain aconsistent appearance of a floating image as the position of theviewer's eyes changes, or to provide different floating images as theposition changes (e.g., to provide a three-dimensional floating image orto increase the apparent three-dimensional structure of athree-dimensional floating image), among other purposes. Alternativelyor additionally, the z-distance of optical element(s) such as FT array604 relative to an image source may be varied (e.g., via actuator 618)in response to changes in viewer position. Alternatively oradditionally, parts of an encoded image that will not reach the viewer'seyes may be deactivated to save power. Further, display device operationmay be modified in response to other events identified via the imagesensor, including but not limited to hand, head, and/or other bodygestures made by a viewer.

As described above, an image source may include a plurality of encodedregions from which encoded image light is output, which may be decodedby suitable optical element(s) such as FT array 604 to produce decodedimage light that forms a floating image viewable from a plurality ofvantage points. As one example, FIG. 12 shows an example decoded image1200 that may be formed by decoding a corresponding encoded image 1300shown in FIG. 13. Encoded image 1300 includes a plurality of encodedregions such as encoded region set 1302, which includes six rows and sixcolumns of encoded regions for a total of thirty-six encoded regions. Amagnified view of encoded region set 1302 is shown in FIG. 14.

Continuing with this example, and with reference to FIG. 6, encodedregion 1302 may correspond to encoded region 606A from which decodedimage light is produced to form the portion 617A of the floating imageviewable from first and second vantage points 614 and 616. As may beinferred from FIGS. 13 and 14, the portion 617A of the floating image isa reflected version of first encoded region 606A and/or encoded region1302. Generally, portions of a floating image may be related tocorresponding encoded regions of an encoded image source by one or moregeometric transformations, including but not limited to translation,reflection, rotation, and/or dilation. Further, as shown in FIGS. 13 and14, one or more encoded regions may include a portion, but not theentirety, of the corresponding image formed by decoded light producedwith encoded image light from the one or more encoded regions. As aparticular example, encoded image 1300 is encoded in a manner to endowimage 1200 with an apparent positive z-distance of 25 mm.

While encoding may be used to make content appear in front, at, orbehind an image source, encoding may be configured to produce a 3Dreconstruction that appears floating. View angles that have overlappingcontent at different heights may be set to display content nearest theviewer for occluded objects, and twice as bright for content objectsthat are transparent (e.g., multiple sources, such as a wire-meshcontent might look twice as bright). A floating image with an apparent3D structure may also be provided via two-dimensional encoding.

In order to achieve a display device operable to produce floating imageswith high resolution, a high resolution display configured to providelight to a lens system may be used. This in turn may stipulate a smallpixel size of the high resolution display to enable a high degree ofgranularity for achieving high resolution output. Alternatively, two ormore displays may be tiled to achieve high resolution output. In eithercase, additional cost and complexity may be imposed on an optical systemintegrating such display(s) by the high resolution stipulation. However,by configuring certain optical properties of a display relative to alens array, high resolution output (e.g., high resolve, sharpness) maybe achieved by associating output from sets of multiple display pixelswith corresponding lenslets of a lens array, without requiring thedisplay itself to have a relatively high resolution.

FIG. 15 shows an example display device 1500 configured to produce ahigh resolution floating image 1502 by associating output from sets ofmultiple display pixels (e.g., encoded regions) with correspondinglenslets of an in-tandem pair of lens arrays 1503. In particular, a sumof a distance d₂ between adjacent lenslets 1504 and an offset Δ is equalto an integer multiple i of a distance p between adjacent pixels 1506.In this example, the maximum view angle θ_(v) is determined by theacceptance NA of lens arrays 1503:

${\theta_{v} = {{\pm \left( \frac{180}{\pi} \right)}{\tan^{- 1}\left\lbrack \frac{d}{2f_{air}} \right\rbrack}}},$

and the focal length in air may be defined in terms of the radius ofcurvature R and refractive index n:

$f_{air} = {\left( \frac{R}{n - 1} \right).}$

The pixel size p₂ (e.g., perceived pixel size) of floating image 1502can be defined in terms of the object pixel pitch p, focal lengthf_(air), and image distance z_(i) (or s_(i)):

$p_{2} = {\frac{s_{i}p}{f_{air}}.}$

Continuing with this example, the offset Δ due to the floating image ats_(i) is

${\Delta = \frac{f_{air}d}{s_{i}}},$

and by stipulating the integer multiple relation described above:d+Δ=ip, the following relation is implied:

${d = \frac{ip}{\left( {1 + {f_{air}\text{/}s_{i}}} \right)}},{{{and}\mspace{14mu} N_{2}} = {\frac{f_{air}N}{s_{i}}.}}$

Addressability can then be estimated based on input object resolutionalong a dimension, focal length, and target image distance s_(i). Itshould be noted by relation between lenslet pitch d and pixel spacing p,that lenslet pitch d may not be an integer multiple of pixel spacing p.For scenarios of forming a two-dimensional floating image, this relationmay provide high resolve of the image detail of sub-pixel content, eventhough addressability would be limited to image addressable resolutionN₂.

One or more of the above-described lens systems and/or display systems(or portions thereof) may be implemented in connection with digitalintegral imaging, e.g., digital integral displays. The digital integralimaging may be used to provide signs, live displays, near-eye displays,light field displays, and other imaging. Light field photographs,integral displays, and related encoding may be provided via theabove-described lens systems.

FIG. 16 depicts an example of telecentric correction via an inner lensarray. For a given output cone NA emitted at a surface normal from adisplay, the amount of light accepted by a single MLA may vary withpixel position, because there may be a roll-off associated with thedisplay view angle. When an FT-pair is used, however, normal exitinglight may be redirected to efficiently pass through the outer lenslet,independent of pixel position. As a result, the effect of intensityroll-off vs view angle may be avoided. This also helps maintain imageformation, as loss of spatial frequency content may be reduced.

FIG. 17 shows an example display device 1700 including a GRIN microlensarray structure 1702 positioned to receive encoded image light from adisplay 1704, which serves as an FT array. Display 1704 includes aplurality of encoded regions, and is spaced away from GRIN structure1702 by a distance z_(g2), which represents the optical gap that enablesoptimal reconvergence at an image conjugate distance z₂. Distance z₂ maysupport a range of z-distances throughout which desired reconvergenceand production of floating imagery is supported, and, as such, may be arange throughout which two and three-dimensional floating images can beproduced. Each cell within GRIN structure 1702 may have a parabolicrefractive index profile to enable each cell to function as an FTequivalent length, and thereby obtain desired image conjugate formationfor each cell and convergence of imaging from multiple cells, along withappropriate encoded content from the encoded regions of display 1704. Asan example, each cell may have an optical length, as defined in terms ofa GRIN lens pitch, of 0.25 p, thus ¼^(th) of the GRIN lens lengthassociated with a full sinusoidal period of cyclic convergence anddivergence. As described above, EMA may be used to block lightencountering the edge of a cell in GRIN structure 1702.

FIG. 18 shows an example display device 1800 including an FT array 1802formed by an in-tandem pair of microlens arrays positioned to receiveencoded image light from a display 1804. Display 1804 includes aplurality of encoded regions, and is spaced away from FT array 1802 by adistance z_(g2), which represents the optical gap that enables optimalreconvergence at an image conjugate distance z_(i), as described above.FT array 1802 includes an in-tandem pair of microlens structures, whichimplement a Fourier transform to decode encoded image light receivedfrom display 1804 and produce two or three-dimensional floating images.In the example depicted in FIG. 18, a first microlens structure exhibitsan array pitch d_(2b), and is positioned optically upstream of a secondmicrolens structure which exhibits an array pitch d₂. The array pitchesd₂ and d_(2b) may be substantially equal or unequal, as required to meetequation relations of array-based imaging to form an image.

In some implementations, an image source may include encoded regionswhich are optically encoded, as opposed to digitally encoded by use of a2D display. For example, a holographic imaging lightguide, which acceptspupil space image content at edge, which may include projected contentfrom a scanned-beam display or projection display, thus may have imagecontent coupled in angle space into the guide, and further include aholographic extraction layer on one side of the guide which extractslight at angles which varies across position along the guide, the Braggresponse being significantly wide for all positions such that a lensarray or FT array, disposed just after the holographic extraction layer,includes lenslets, each of which may image the angular light contentinto position space to fill the width of the lenslet, and may thusprovide an optically encoded image source which may be used as input tothe FT array, in order to decode the encoded image light into a floatingimage. The angular acceptance of the holographic imaging guideextraction layer substantially matches the angular acceptance of a lensarray or FT array which is used to form the optical encoding imagesource. As an example illustrating the utilization of optically encodedregions, FIG. 19 shows an example display device 1900 includingarray-based imaging layers 1902, an optical encoding layer 1904, and anextraction layer 1906. Optical encoding layer 1904 may include a lens orFT array, for example.

In some embodiments, the methods and processes described herein may betied to a computing system of one or more computing devices. Inparticular, such methods and processes may be implemented as acomputer-application program or service, an application-programminginterface (API), a library, and/or other computer-program product.

FIG. 20 schematically shows a non-limiting embodiment of a computingsystem 1900 that can enact one or more of the methods and processesdescribed above. Computing system 1900 is shown in simplified form.Computing system 1900 may take the form of one or more personalcomputers, server computers, tablet computers, home-entertainmentcomputers, network computing devices, gaming devices, mobile computingdevices, mobile communication devices (e.g., smart phone), and/or othercomputing devices.

Computing system 1900 includes a logic machine 2002 and a storagemachine 2004. Computing system 1900 may optionally include a displaysubsystem 2006, input subsystem 2008, communication subsystem 2010,and/or other components not shown in FIG. 20.

Logic machine 2002 includes one or more physical devices configured toexecute instructions. For example, the logic machine may be configuredto execute instructions that are part of one or more applications,services, programs, routines, libraries, objects, components, datastructures, or other logical constructs. Such instructions may beimplemented to perform a task, implement a data type, transform thestate of one or more components, achieve a technical effect, orotherwise arrive at a desired result.

The logic machine may include one or more processors configured toexecute software instructions. Additionally or alternatively, the logicmachine may include one or more hardware or firmware logic machinesconfigured to execute hardware or firmware instructions. Processors ofthe logic machine may be single-core or multi-core, and the instructionsexecuted thereon may be configured for sequential, parallel, and/ordistributed processing. Individual components of the logic machineoptionally may be distributed among two or more separate devices, whichmay be remotely located and/or configured for coordinated processing.Aspects of the logic machine may be virtualized and executed by remotelyaccessible, networked computing devices configured in a cloud-computingconfiguration.

Storage machine 2004 includes one or more physical devices configured tohold instructions executable by the logic machine to implement themethods and processes described herein. When such methods and processesare implemented, the state of storage machine 2004 may betransformed—e.g., to hold different data.

Storage machine 2004 may include removable and/or built-in devices.Storage machine 2004 may include optical memory (e.g., CD, DVD, HD-DVD,Blu-Ray Disc, etc.), semiconductor memory (e.g., RAM, EPROM, EEPROM,etc.), and/or magnetic memory (e.g., hard-disk drive, floppy-disk drive,tape drive, MRAM, etc.), among others. Storage machine 2004 may includevolatile, nonvolatile, dynamic, static, read/write, read-only,random-access, sequential-access, location-addressable,file-addressable, and/or content-addressable devices.

It will be appreciated that storage machine 2004 includes one or morephysical devices. However, aspects of the instructions described hereinalternatively may be propagated by a communication medium (e.g., anelectromagnetic signal, an optical signal, etc.) that is not held by aphysical device for a finite duration.

Aspects of logic machine 2002 and storage machine 2004 may be integratedtogether into one or more hardware-logic components. Such hardware-logiccomponents may include field-programmable gate arrays (FPGAs), program-and application-specific integrated circuits (PASIC/ASICs), program- andapplication-specific standard products (PSSP/ASSPs), system-on-a-chip(SOC), and complex programmable logic devices (CPLDs), for example.

The terms “module,” “program,” and “engine” may be used to describe anaspect of computing system 1900 implemented to perform a particularfunction. In some cases, a module, program, or engine may beinstantiated via logic machine 2002 executing instructions held bystorage machine 2004. It will be understood that different modules,programs, and/or engines may be instantiated from the same application,service, code block, object, library, routine, API, function, etc.Likewise, the same module, program, and/or engine may be instantiated bydifferent applications, services, code blocks, objects, routines, APIs,functions, etc. The terms “module,” “program,” and “engine” mayencompass individual or groups of executable files, data files,libraries, drivers, scripts, database records, etc.

It will be appreciated that a “service”, as used herein, is anapplication program executable across multiple user sessions. A servicemay be available to one or more system components, programs, and/orother services. In some implementations, a service may run on one ormore server-computing devices.

When included, display subsystem 2006 may be used to present a visualrepresentation of data held by storage machine 2004. This visualrepresentation may take the form of a graphical user interface (GUI). Asthe herein described methods and processes change the data held by thestorage machine, and thus transform the state of the storage machine,the state of display subsystem 2006 may likewise be transformed tovisually represent changes in the underlying data. Display subsystem2006 may include one or more display devices utilizing virtually anytype of technology. Such display devices may be combined with logicmachine 2002 and/or storage machine 2004 in a shared enclosure, or suchdisplay devices may be peripheral display devices.

When included, input subsystem 2008 may comprise or interface with oneor more user-input devices such as a keyboard, mouse, touch screen, orgame controller. In some embodiments, the input subsystem may compriseor interface with selected natural user input (NUI) componentry. Suchcomponentry may be integrated or peripheral, and the transduction and/orprocessing of input actions may be handled on- or off-board. Example NUIcomponentry may include a microphone for speech and/or voicerecognition; an infrared, color, stereoscopic, and/or depth camera formachine vision and/or gesture recognition; a head tracker, eye tracker,accelerometer, and/or gyroscope for motion detection and/or intentrecognition; as well as electric-field sensing componentry for assessingbrain activity. For example, in some embodiments, a depth sensor, suchas time of flight (TOF) or stereo depth, providing a depth map of ascene, may be used to track a user, gestures, and/or head movements, andprovide changes to encoded content in order to provide a floatingdisplay which is interactive with the user.

When included, communication subsystem 2010 may be configured tocommunicatively couple computing system 1900 with one or more othercomputing devices. Communication subsystem 2010 may include wired and/orwireless communication devices compatible with one or more differentcommunication protocols. As non-limiting examples, the communicationsubsystem may be configured for communication via a wireless telephonenetwork, or a wired or wireless local- or wide-area network. In someembodiments, the communication subsystem may allow computing system 1900to send and/or receive messages to and/or from other devices via anetwork such as the Internet.

Another example provides a display device comprising an image sourceincluding a plurality of encoded regions from which encoded image lightis output, and a Fourier transform array positioned to receive theencoded image light and output decoded image light that forms a floatingimage viewable from a plurality of different vantage points, whereinfrom a first vantage point decoded image light forming a portion of thefloating image originates from a first encoded region, and wherein froma second vantage point decoded image light forming the portionoriginates from a second encoded region, different than the firstencoded region. In such an example, the image source alternatively oradditionally may include a display image. In such an example, the imagesource alternatively or additionally may include a static image. In suchan example, the image source alternatively or additionally may include astitched display image formed by optically combining output from two ormore displays. In such an example, the output from the two or moredisplays alternatively or additionally may be optically combined via alens array. In such an example, the output from the two or more displaysalternatively or additionally may be optically combined via array-basedimaging. In such an example, the Fourier transform array alternativelyor additionally may include a lens array pair. In such an example, theFourier transform array alternatively or additionally may include agraded-index microlens array structure. In such an example, the floatingimage alternatively or additionally may be a three-dimensional image. Insuch an example, the floating image alternatively or additionally may bea two-dimensional image. In such an example, the floating imagealternatively or additionally may have an apparent z-distance that ispositive relative to the image source. In such an example, the floatingimage alternatively or additionally may have an apparent z-distance thatis negative relative to the image source. In such an example, thefloating image alternatively or additionally may have an apparentz-distance that is coplanar with the image source. In such an example,the portion of the floating image alternatively or additionally may berelated to the first encoded region by a geometric transformation. Insuch an example, the display device alternatively or additionally maycomprise an actuator operatively coupled to the Fourier transform arrayand configured to adjust a position of the Fourier transform arrayrelative to the image source to thereby adjust an apparent z-distance ofthe floating image. In such an example, the image source alternativelyor additionally may be configured to vary the encoded image light tothereby adjust an apparent z-distance of the floating image. In such anexample, the floating image alternatively or additionally may have anapparent z-distance that is determined at least in part by a pitch ofthe plurality of encoded regions relative to a pitch of the Fouriertransform array.

Another example provides a display device comprising an image sourceincluding two or more displays that cooperatively form a stitcheddisplay image including a plurality of encoded regions of encoded imagelight, and a Fourier transform array positioned to receive the encodedimage light and output decoded image light that forms a floating imageviewable from a plurality of different vantage points, wherein from afirst vantage point decoded image light forming a portion of thefloating image originates from a first encoded region, and wherein froma second vantage point decoded image light forming the portionoriginates from a second encoded region, different than the firstencoded region. In such an example, the floating image alternatively oradditionally may be a two-dimensional image, and the two-dimensionalimage alternatively or additionally may have an apparent z-distance thatis positive relative to the image source.

Another example provides a display device comprising an image sourceincluding a plurality of encoded regions from which encoded image lightis output, and a lens array including a plurality of lenslets, the lensarray positioned to receive the encoded image light and output decodedimage light that forms a floating image viewable from a plurality ofdifferent vantage points, wherein from a first vantage point decodedimage light forming a portion of the floating image originates from afirst encoded region, wherein from a second vantage point decoded imagelight forming the portion originates from a second encoded region,different than the first encoded region, and wherein a sum of a distancebetween adjacent lenslets and an offset is equal to an integer multipleof a distance between adjacent encoded regions.

It will be understood that the configurations and/or approachesdescribed herein are exemplary in nature, and that these specificembodiments or examples are not to be considered in a limiting sense,because numerous variations are possible. The specific routines ormethods described herein may represent one or more of any number ofprocessing strategies. As such, various acts illustrated and/ordescribed may be performed in the sequence illustrated and/or described,in other sequences, in parallel, or omitted. Likewise, the order of theabove-described processes may be changed.

The subject matter of the present disclosure includes all novel andnon-obvious combinations and sub-combinations of the various processes,systems and configurations, and other features, functions, acts, and/orproperties disclosed herein, as well as any and all equivalents thereof.

1. A display device, comprising: an image source including a pluralityof encoded regions from which encoded image light is output; and aFourier transform array positioned to receive the encoded image lightand output decoded image light that forms a floating image viewable froma plurality of different vantage points, wherein from a first vantagepoint decoded image light forming a portion of the floating imageoriginates from a first encoded region, and wherein from a secondvantage point decoded image light forming the portion originates from asecond encoded region, different than the first encoded region.
 2. Thedisplay device of claim 1, wherein the image source includes a displayimage.
 3. The display device of claim 1, wherein the image sourceincludes a static image.
 4. The display device of claim 1, wherein theimage source includes a stitched display image formed by opticallycombining output from two or more displays.
 5. The display device ofclaim 4, wherein the output from the two or more displays is opticallycombined via a lens array.
 6. The display device of claim 4, wherein theoutput from the two or more displays is optically combined viaarray-based imaging.
 7. The display device of claim 1, wherein theFourier transform array includes a lens array pair.
 8. The displaydevice of claim 1, wherein the Fourier transform array includes agraded-index microlens array structure.
 9. The display device of claim1, wherein the floating image is a three-dimensional image.
 10. Thedisplay device of claim 1, wherein the floating image is atwo-dimensional image.
 11. The display device of claim 1, wherein thefloating image has an apparent z-distance that is positive relative tothe image source.
 12. The display device of claim 1, wherein thefloating image has an apparent z-distance that is negative relative tothe image source.
 13. The display device of claim 1, wherein thefloating image has an apparent z-distance that is coplanar with theimage source.
 14. The display device of claim 1, wherein the portion ofthe floating image is related to the first encoded region by a geometrictransformation.
 15. The display device of claim 1, further comprising anactuator operatively coupled to the Fourier transform array andconfigured to adjust a position of the Fourier transform array relativeto the image source to thereby adjust an apparent z-distance of thefloating image.
 16. The display device of claim 1, wherein the imagesource is configured to vary the encoded image light to thereby adjustan apparent z-distance of the floating image.
 17. The display device ofclaim 1, wherein the floating image has an apparent z-distance that isdetermined at least in part by a pitch of the plurality of encodedregions relative to a pitch of the Fourier transform array.
 18. Adisplay device, comprising: an image source including two or moredisplays that cooperatively form a stitched display image including aplurality of encoded regions of encoded image light; and a Fouriertransform array positioned to receive the encoded image light and outputdecoded image light that forms a floating image viewable from aplurality of different vantage points, wherein from a first vantagepoint decoded image light forming a portion of the floating imageoriginates from a first encoded region, and wherein from a secondvantage point decoded image light forming the portion originates from asecond encoded region, different than the first encoded region.
 19. Thedisplay device of claim 18, wherein the floating image is atwo-dimensional image, and wherein the two-dimensional image has anapparent z-distance that is positive relative to the image source.
 20. Adisplay device, comprising: an image source including a plurality ofencoded regions from which encoded image light is output; and a lensarray including a plurality of lenslets, the lens array positioned toreceive the encoded image light and output decoded image light thatforms a floating image viewable from a plurality of different vantagepoints, wherein from a first vantage point decoded image light forming aportion of the floating image originates from a first encoded region,wherein from a second vantage point decoded image light forming theportion originates from a second encoded region, different than thefirst encoded region, and wherein a sum of a distance between adjacentlenslets and an offset is equal to an integer multiple of a distancebetween adjacent encoded regions.